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ABSTRACT 

The purpose of this three-year project is to 
impleiuent elementary music programs in the public schools of the 
region by operating an exemplary center in music education, and by 
establishing procedural patterns for school systems to follow. 
Standardized and locally constructed tests and surveys show rapid 
progress of students and significant attitude change of teachers, 
administrators and parents. The methods of evaluating the objectives 
range from simple to nearly impossible. The definitive evaluation of 
project success will depend on the increase in authorized and funded 
music teacher positions* (Author) 
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Abstract 

The purpose of the three year project is to imple- 
ment elementary nrusic programs in the public schools of 
the region by operating an exemplary center in music 
education and establishing procedural patterns for school 
systems to follow. Standardized and locally constructed 
tests and surveys show rapid progress of students and 
significant attitude change of teachers, administrators 
and parents. The problems of evaluating the objectives 
range from simple to near impossible. The definitive 
evaluation of project success will depend on the increase 
in authorized and funded music teacher positions within 
the project region. 



Introduct ion 

This paper reports on the problems involved in 
the process of formative evaluation in a music program in 
Region Eight of Tennessee. The overall purpose of the 
project is to bring about significant constructive change 
in the. scope and operation of elementary music programs 
in the public schools of that region. The goals are to 
establ::.sh and operate an exemplary center in music educa- 
tion ard to establish procedural patterns for school 
systems to follow in their commitment to provide quality 
instructional programs in music. This Implementing Elemen- 
tary Music Improvement (I. E.M.I.) is now in its third and 
final year. It is funded by E.S.E.A. Title III through 
the Tennessee State Department of Education. 

The I. E.M.I, project is so extensive this paper 
will report on only some of the evaluation problems. As 
usual, formulating testable criteria for the several 
objectives of tlie project has been a problem. Also, 
internal formative evaluation was not a part of the original 
proposal and was added in the second year of the project. 
Fortunately, base line data were collected and are being 
used in the evaluative proce s s . 
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As reported by Grobman (1970)5 tests can touch on 
only some of the aspects of evaluation. In fact, evaluation 
is equated with testing but the use of standardized tests 
does not guarantee an adequate evaluation. Seldom does 
a nationally standardized test measure local objectives , 
and this is certainly true in this project. 

The technique used pr imarl ly by the project 
personnel for music instruction is called Orf f -Schulwerk . 
This system, described by Reimer, 1970; Walter, 1969; and 
Woodruff, 1970, is an innovative method of providing young 
children with the basic fundamentals of music. It is 
concerned with the coordination of movement with free 
expression and is designed to begin very early before a 
child is confronted with traditional instruction on regular 
musical instruments . 

The purposes and objectives of the project are as 

follows : 

Purpose 1 to establish and operate an exemplary center 

in music education for children in the elementary grades. 

Ob jectives : 

]. In the cognitive domain each student will gain 

the knowledge n-^eded to achieve the behavioral items 
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of the Music Behavioral Expectations Guide List 
(MBEGL) for his grade level. 

2. In the affective domain each student will acquire 
value of particip^'^tion in music and will respond 
positively toward music and toward the music 
instructional program . 

3. In the psychomotor domain each student will develop 
the physical skills needed to achieve the behavioral 
i tems of the MBEGL for his grade level . 

Purpose 2 to establish procedural patterns for school 

systems to follow in their commitment to provide quality 
instructional programs in music at the elementary school 
level. 

Ob Jec 1 1 ves : 

1- Music teachers and classroom teachers possessing 
musical backgrounds and participating in the in- 
service education sessions will acnuire knowledge 
and sKills sufficient to begin programs of similar 
intent and extent. 

2. Through in-service education sessions, classroom 
teachers with limited musical backgrounds will 
dovel op musical skills and acquire understandings 
of the purposes of a good music program to the 
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extent that they will be sympathetic to such 
programs, assist in the establishment of such 
programs within their respective schools and 
school systems, and incorporate valid concepts 
of music in their classroom teaching. 

3 . Administrators participating in the in-service 
education sessions will acquire an understanding 
of the intent of the project, value the benefits 
of a good program of music for children and will 
assist in setting up similar programs in their 
school systems . 

4. Children, their teachers, parents and community 
will acquire an understanding of the nature of a 
general-vocal music program, the teaching procedures, 
material and equipment employed by the Project. 

5. Children, their teachers, parents and community 
will become aware of the potential impact of an 
active general-vocal music program by having 
opportunities to see, hear and meet visiting 
performing musicians. 

Ob ject ive added to the las t year of the project : 

Classroom teachers in the project region in grades 
K-3 and grades will present music instruction 

to their student s using a program structure suggested 
Q bv the project. 
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Through this in-service effort, classroom teachers 
with limited musical backgrounds will develop 
musical skills and acquire understandings of the 
purposes of a good music program to the extent 
that they will be sympathetic to such programs, 
assist in the establishment of such programs within 
their respective schools and school systems and 
incorporate valid concepts of music in their class- 
room teaching. 

Tests 

The primary method of assessing cognitive change 
in music ability of the students in thv. demonstration school 
was repeated administrations of the ^\ls^c Achievement Test 
(MAT), (ColwelL, 1969). The MAT consists of four tests 
each having several sub-tests. It provides base line and 
growth data in several major areas thought important for 
a musically knowledgable person. 

The manual reports that a score du the MAT will 
provide the following information on each student: 

(1) A measure of the extent to which a pupil has 
profited from past musical instruction. 

(2) A measure of the quality of his musical instruc- 
tion. 
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(3) An indication of the extent to which the pupil 
is likely to profit from further nrusical 
instruction. 

The statistical data developed by the standardi- 
zation process provides normative data for students in 
grades four through twelve . 

Only three of the four tests of the MAT have been 
used. Test No. 1 covers the areas of pitch, interval and 
meter discriminations. Test No. 2 covers the areas of 
major-minor mode discrimination, feeling for tonal center 
and auditory-visual discrimination. Test No. 3 covers the 
areas of tonal memory, and recognition of melody, pitch 
and musical instriament s . Test No. 4, which has not been 
administered, covers the area;^ of musical style, auditory — 
visual discrimination, chord recognition, and cadence 
recognition. 

The administration of the MAT is accomplished using 
publ i shod answer sheets and recordings . Answer sheets are 
hand scored using templates. 

These annual administrations of the MAT have been 
the most precise data collected thus far. The gains 
reported are for 2 separate groups. When the project began, 
the 4th and 5th grades in the demonstration school started 
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at the same t ime . We were able , then, to study the 
effects of early and late starting of music instruction. 

Table 1 shows the 3 administrations of the MAT for 
group 1. Group 1 includes those students who started their 
mvisic instruction in the 4th grade . Table 2 shows the MAT 
results for Group 2 who started in the 5th grade. 

Table 1 shows the average 3 test gain from the 1st 
to the 2nd year was 15.47. This is contrasted with the 
average gain of the standardized national sample of 9.00. 
Progress continued during the 2nd year but not at the same 
accelerated rate. 

Table 2 shows group 2 also growing at an accelerated 
rate and closing on the means of the national sample. The 
3 test gain for the 2 year period was 23.96 for the students 
in the demonstration school as opposed to the national 
average gain of 20.74. As in other educational studies of 
various grade levels, greater gains were made in the lower 
than upper grades. 

Using the increase in means from year to year as 
reported in the national norms as a ^^control^' group is not 
the most des irable si tuat ion . However , i t does give an 
indication of ^^normal growth^^ in musical achievement. Since 
the increase in scores from the protest to posttest was 
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more than the nat ional norin increase , it can be concluded 
that the musical instruction received by the pupils has 
resulted in accelerated growth. 

The MAT scores from the demons t rat ion school were 
also compared with o^.her data. MAT tests 1 and 2 were 
administered to the 5th and 6th grades of 5 other schools. 
Three schools were from within the project region and had 
very little organized music instruction. Two other schools 
were randomly selected from outside the project -region. 

Table 3 shows a comparison among the National Norms, 
the demonstration school and 5 other ochools in regard to 
tests 1 and 2 of the MAT. The L --ratios given are for theee 
differences between the demonstration school and the 5 
other schools column. 

Within the 5th grade there was not much difference 
between the demonstration group and the 5 other schools for 
test 1 . However , test 2 showed the demons t rat ion school 
significantly higher (p<.Ol) than the 5 other schools. 

On the 6th grade level , the demons t rat ion school 
was significantly higher on both tests (p<Coi) . 

Te St 1 did not show any difference between the 
groups on the 5th grade level. However, there was a 
significant difference Cor test 2 (p^.05). Test 2 showed 
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a significant difference between the groups on both the 
5th and 6th grade levels. Perhaps this can be explained 
by the level of difficulty of each test. 

When the 5 other schools were compared, the 3 
schools in the project region were superior on both tests, 
but were not statistically significant. The I,E,M,I, 
schools had total scores of 75.18 and 75.57 for the 5th 
and 6th grades respectively while the outside schools had 
72.09 and 73.51. 

A locally constructed test has been completed 
which is based on items from the Music Behavioral Expecta - 
tions Guide List (MBEGL) , This document was developed by 
music educators and the State Department of Education. 
It lists, in behavioral terms, expected behavior in music 
education for grades K-6. This MBEGL test includes both 
a written and performance section and was administered once 
to a sample of grades 3,4,5, and 6 of the demonstration 
school . 

Table 4 shows the mean scores for each grade and 
the person or correlations between the part scores and 
MAT scores. The correlation between the written and 
performance part of the test was r=.51. The correlations 
with MAT were substantial with only one exception. The 
3rd grade written portion had virtually no relationship 
O with the MAT. 



ERIC 



10 

The performance section of the MBEGL tes'c also 
served as a measure of the psychomotor domain in musical 
achievement . Thi s was an effort to assess Objective 3 
of purpose 1 , 

Surveys 

This section deals with the evaluation of project 
activities using less precise instruments than were possible 
with the cognitive domain. Surveys of two touring programs, 
a monthly music instruction guide and an attitude survey 
of a provi sions statement , as reported here , are only three 
examples of the many activities of the project. 

Two special tor.ring programs were sponsored by the 
project for the following purposes: 

(1) to increase the projects' visibility in the 
participating communities and throughout 
Tennessee 

(2) to bring to the residents of the project region 
out-s taiiding examples of music and to expose 
the residents, especially the children, to 
exciting, entertaining education experiencies 
of high musical quali t y 

(3) to stimulate the implementation of a music 
program, or the improvement of an existing 
program in the participating school districts. 
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The special programs were week-long tours of the 
project region by Miss Ella Jenkins, a folk singer special- 
izing in participatory programs for children, and by the 
Chicago Childrens Choir, a forty-two member group of boys 
and girls ages 10-18. 

Miss Jenkins performed twice daily among all 
communities in the project region. The programs were held 
in the largest available facility. Children were bussed 
in from all or most schools of the system to provide the 
largest audience possible. She performed with more than 
14,000 children during the week. 

Table 5 shows the results of 395 questionnaires 
administered to teachers during Miss Jenkins' program. The 
overall percentage of favorable responses to Miss Jenkins 
was 92.37o. Neutral and negative responses resulted in a 
percentage of 2.17= while 5 . 67o of the questions were unanswered. 

Table 6 shows the results of 399 returned question- 
naires filled out by adult visitors who were not teachers. 
The high percentage of favorable responses in all counties 
indicates the consistent high quality of her performances. 

The tour by The Chicago Children's Choir took place 
during the first week of April, 1973. The choir performed 
in every school district of the project region traveling 
via chartered bus and making morning and afternoon appear- 
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ancGS each day of the week. In aadition, the choir sang 
three evening programs that were open to the general public. 
A teachers workshop concluded the choir's tour week. 

Like Miss Jenkins' tour, the choir was preceded by 
a pre tour information packet and recordings provided by 
the project. The local schools again bussed students from 
throughout their systems to central auditoriums. At one 
program, the choir performed for 4,200 children. Total 
attendance for the choir^s week exceeded 13,500. The 
communities provided home hospitality for the choir members. 

A auestionnaire similar to the one used for Miss 
Jenkins' tour was administered to 377 teachers. The overall 
percenta^re of favorable responses for all counties was 85.l7o. 
Responses considered neutral or unfavorable reached 8.47o. 
Items not answered were 5.37o. 

Both tours were evaluated using the survey question- 
naires, written comments on the questionnaires, observations 
by teachers, and attendance records. The statistical 
summaries of these instruments were limited, but written 
comiments bv the respondents were of value. However self- 
serving these qiies t ionnaires seemed to be, they did provide 
good feedback to the project personnel. 

A sub-project within the project is a monthly 
publication called Music Time . It is composed of small 
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music lessons that are topical and can be used by regular 
classroom teachers. Two levels. Primary and Intermediate , 
are prepared and distributed monthly to instructional 
personnel in the ten school districts participating in the 
I.E.M,I. project. The distribution reaches 80 schools, 
1,035 teachers, 18 music specialists, 12 supervisors and 
10 superintendents. These school personnel have regular 
contact with more than 33,120 children in grades K'6. 

In an effort to determine the value and effective- 
'^.ess of Music Time , a questionnaire was administered with 
the November, 1973 bulletin. Table 7 shows the abbreviated 
results of 254 returned questionnaires. 

The simple questions furnished good feedback to 
the project personnel. The various comments and suggestions 
also greatly aided in the composition of subsequent issues. 

Some concrete evidence evaluating the overall 
effectiveness of the project was gathered after the first 
year. The administrators (superintendents, supervisors and 
principals) were asked to endorse a statement of Elementary 
Mtisic Education Provisions which called for: 

(1) The development of a curriculum guide 

(2) A specialist approach to instruction 

(3) A maximum pupil load per week of 400 

(4) A minimum of 75 minutes per week of music 
instruction 

(5) A music room for instruction 

(6) Provision of materials and equipment 
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Of the sixty-seven principals responding, 86.6 percent 
endorsed the Provisions Statement by answering ''Yes'' on a 
questionnaire. The remaining at least "partly" endorsed the 
statement. About three-quarters reported that they would 
recommend the hiring of a full time, qualified music teacher 
and that a music instruction room be assigned. Ninety-four 
percent reported that they would recommend the acquisition 
of instruments, equipment and materials. 

At the termination of the program another survey 
will be conducted to determine exactly what each school 
system plans for the future. Actual positions created, 
equipment purchased, and number of designated music class- 
rooms will be the most objective evaluation of the success 
of the project. 

Discus si on and Summary 

The purposes and goals of the project are con'^tantly 
being evaluated at various levels. Some ill-conceived 
instruments have been discarded and others have been radic- 
ally revised. Many activities occured only once and oppor- 
tunities for improving evaluation instruments were limited. 

This project, as others, where people must work 
together, generated some conflict. Early in the project 
friction among some personnel almost became critical. 
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However, adjustments in job assigp'-nent s greatly alleviated 
the stress of this problem. 

The intrus ion of "out s ider s*' into the ^ural area 
of the project region continues to be a problem. The 
cooperation of superintendents and principals has helped, 
but some teachers remain suspicious. Job insecurity and 
competition for classrooms has been a continuing problem. 
One of the important provisions statements is concerned 
with the assignment of classrooms for music instruction 
only and apparently some teachers think they might loose 
their rooms to a music teacher. 

This project demonstrates well the varied means of 
evaluating educational objectives. The technique of 
assessing some objectives seems readily apparent and simple 
to administer while others appear to be impossible to 
evaluate. The musical abilities of children can be reason- 
ably ascertained with pre and posttests. However, to 
accurately measure a change in enthusiasm or attitudes 
toward music instruction in a school system must wait until 
the completion of the program. If there is a significant 
increase in authorized and funded music teacher positions 
at that time, it would be an accurate measure of project 
success. Meanwhile, principals and superintendents can 
only promise to approach their school boards for the 
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additional appropriations. Hopefully the problems and 
solutions reported in this study will help others who have 
the task of evaluating similar projects. 
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Table 1 

Repeated administrations. National norms and gains 
of >Kisic Achievement Tests ^ 1, 2, and 3 by grade level for 
group 1 . 



TEST 1 

Means Gains 



Grade 




4 


5 




6 


4- 


-5 


5- 


-6 


4- 


-6 


Demo 


34 


. 61 


38. 


, 35 


42 


.43 


3. 


, 74 


4, 


.08 


7. 


,82 


Nat ional 


45 


.02 


47. 


,43 


50 


.31 


2. 


,41 


2, 


,88 


5. 


,29 












TEST 2 














Demo 


31 


.54 


37, 


,85 


40 


.89 


6, 


,31 


3, 


.04 


9, 


,35 


National 


36 


.86 


40, 


, 22 


43 


.10 


3. 


,36 


2, 


,88 


6, 


,24 












TEST 3 














Demo 


20 


. 53 


25. 


,95 


26 


.81 


5, 


,42 




,86 


6. 


,28 


National 


19 


.86 


23. 


,09 


27 


.17 


3. 


,23 


4, 


,08 


7, 


,31 



THREE TEST TOTAL 



Demo 86.68 102.15 110.13 15.47 7.98 23.45 

National 101.74 110.74 120.58 9.00 9.84 18.84 



Table 2 

Repeated administrations. National Norms and gains 
of Music AsChievement Tests 1, 2 and 3 by grade level for 
Group 2 . 







Means 








Gains 






Grade 


5 


6 


7 




5- 


-6 


6- 


-7 


5- 


."7 












TEST 1 














Demo 


33, 


,75 


37. 


,94 


41. 


41 




, 19 


3. 


,47 


■7 
/ . 


, DO 


National 


47. 


,43 


50. 


,31 


51. 


35 


2. 


,88 


1. 


,04 


3. 


, 92 












TEST 2 














Demo 


31, 


,04 


35. 


, 73 


42. 


17 


4 


69 


6. 


,44 


11. 


. 13 


National 


40, 


,22 


43. 


,10 


47. 


56 


2. 


,88 


4. 


,46 


7, 


, 34 












TEST 3 














Demo 


20, 


, 70 


24. 


10 


25. 


87 


3. 


,40 


1. 


,77 


5, 


17 


National 


23. 


,09 


27. 


,17 


32. 


57 


4. 


,08 


5. 


,40 


9. 


,48 



THREE TEST TOTAL 



Demo 85.49 97.77 109.45 12.28 11.68 23.96 

National 110.74 120.58 131.48 9.84 10.90 20.74 



Table 3 

5th and 6th grade scores of MAT tests 1 and 2 from 5 schools 
compared with National Norms and 5th and 6th grade scores 
at the demonstration school. 



GRADE NATIONAL DEMO 5 OTHER t 

NORM SCHOOL SCHOOLS 



BOTH TESTS 

5th 87.65 76.20 73.97 1.76 

6th 93.41 83.32 74.82 2.90 p<.01 



TESTS 1 

5th 47.43 38.35 39.46 .86 

6th 50.31 42.43 39.53 2.25 p<.05 



TESTS 2 



5th 
6th 



40. 22 
43. 10 



37.85 
40.89 



34.51 2.04 p<.05 
35.29 3.41 p<.Ol 



Table 4 

Means and correlations for Music Behavioral 
Expectations Guide List test and the Music Achievement Test . 

MBEGL Test 









Written 




Performance 


Grade 


N 


MAT 


Mean 


r 


Mean 


r 


3 


31 


71.81 


16.35 


.02 


23.75 


.56 


4 


31 


108.44 


13.48 


.61 


20. 71 


.45 


5 


27 


113.12 


28.41 


.69 


17.14 


.54 


6 


26 


100.53 


29.69 


.63 


43.41 


.68 



Table 5 



Percentage of each Response by teachers to questionnaire 

adminis tere d at the Ella Jenkins Concert . 

(l) How would you rate the advance publicity of Ella 
Jenkins* tour? 



Excellent 53.0 

Good 37.8 

Undecided 1 . 3 

Fair 1.8 
Poor 

N.A. 6.4 



(2) How much did you use the advance materials to prepare 
the children? 

Very much 43 . 1 

Some 39.8 

Little 3.6 

Did not use 1 . 6 

Did not receive 3.1 

(3) How do you feel your students anticipated the coming 
of Ella Jenkins' concert? 

Very excited 63 . 1 

Mildly excited 27.9 

Indifferent 1 . 1 
Negative 

N.A. 8.1 



(4) How did you anticipate the coming of the concert? 

Very excited 51.2 

Mildly excited 39.0 

Indifferent . 8 
Negative 

N.A. 9.2 

(5) Do you feel that you increased your musical concepts 
and skills as a result of the concert? 

Definitely yes 28.9 

Yes 54.2 

Undecided 5 . 6 

No 4.1 
Definitely no 

O N.A. 7.1 
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Table 6 



Percentage of each Response by adult visitors to a question- 
naire administered at the Ella Jenkins Concert. 

(1) How would you rate your enjoyment of the program? 

Excellent 75.2 

Good 20.6 

Undecided .8 

Fair 1.0 

Poor 0 

N.A. 2.3 

(2) How do you think the children would rate the program? 

Excellent 79.0 

Good 18.6 

Undecided .5 

Fair .3 

Poor 0 

N.A. 1.8 



(3) How would you rate Ella Jenkins' technique with children? 



Excellent 79.5 

Good 17.3 

Undecided .5 

Fair -8 

Poor 0 

N.A. 2.0 



(A) Did you personally participate in the program with 
Ella Jenkins? 

Throughout 69.0 

Occasionally 24.9 

Not at all ' 3.5 

N.A. 2.8 



(5) Do you feel that the Ella Jenkins tour gives constructive 
publicity to the TEMI project? 



Definitely 61.9 

Yes 32.9 

Undecided 2.5 

No . 3 

Definitely not 0 

N.A. ' 2.5 



Table 7 



Percent of each response for Music Time nuestionnaire . 



1. ...receive Music Time for September? 



Yes 76.8?. 

No 10 . 67, 

No answer . . . .12.6% 

... for October? 

Yes 82.37o 

No 3 . 97; 

No answer .... 13.87, 

2. . .used ...suggestions with children? 

mch 9.47o 

Some 54 . 77o 

A little .... 20.17o 

Not at all ... 13.07o 

No answer ... 2 . 87o 

3. . . .children enjoy. . . .? 

Very much . . . . 24.07o 

Much 36.27; 

Neutral 19.37o 

Little 5.97o 

Not at all ... 3.57o 

No answer . . . . 11.17o 

4. ...level of musical knowledge...? 

Very low 87o 

Low 5.97o 

About right . . . 71.37o 

High 5.57o 

Too high .... 3 . 17o 
No answer . . . . 14.67o 

5. . . .amount of material . . .7 

Too little . . . 14.67o 

About right ... 70 , 97, 

Too much .... 3 . 57o 

No answer . . . . 14.57, 
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6. ...use more material ...? 

Yes 62.2% 

No 21.3% 

No answer .... 16.57o 

7. ...keep a record...? 

Yes 12.67„ 

No 72.4% 

No answer .... 15.0?o 
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